feat: AWS Security & Compliance — finding-to-code mapping with AI remediation#2282
Merged
feat: AWS Security & Compliance — finding-to-code mapping with AI remediation#2282
Conversation
…egration Add `atmos ai security` and `atmos ai compliance` CLI commands with full AWS Security Hub integration, dual-path finding-to-component mapping (tag-based and heuristic), and multi-format report rendering (markdown/json/yaml/csv). Phase 1: Schema, CLI commands, report renderers, sentinel errors. Phase 2: AWS SDK clients (Security Hub, Resource Groups Tagging API), paginated finding fetcher with filters, batch tag-based mapping (Path A), naming convention and resource type heuristics (Path B), unit tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ce_report Phase 3: Register three new AI tools for the agent to use: - atmos_list_findings: Query Security Hub findings with severity/source/stack filters - atmos_describe_finding: Get detailed info about a specific finding by ID - atmos_compliance_report: Generate compliance posture reports for CIS/PCI/SOC2/HIPAA/NIST Tools are registered as read-only core tools in the AI tool registry, available in both chat and MCP modes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add findings cache with configurable TTL to reduce AWS API calls - Add report renderer tests (17 tests covering all formats) - Add Docusaurus docs for atmos ai security and compliance commands - Increase test coverage from 71% to 91% - Update PRD to support all 7 Atmos AI providers (not just Bedrock) - Position Bedrock for enterprise customers needing data residency Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Implement FindingAnalyzer that sends findings + component source to configured AI provider for root cause analysis and remediation - Parse AI response into structured Remediation (root cause, deploy command, risk level) - Integrate analyzer into security command (runs unless --no-ai is set) - Add ErrAISecurityAnalysisFailed sentinel error - Tests for analyzer, prompt building, response parsing (91.8% coverage) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Register atmos_analyze_finding tool in AI tool system - Fetches finding by ID, maps to component, runs AI analysis - Returns root cause, remediation, deploy command, and risk level - Accepts optional component_source and stack_config for richer context Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move security and compliance commands from `atmos ai` to `atmos aws` since they are AWS-specific and work without AI. AI analysis is now opt-in via the `--ai` flag (default: off) instead of opt-out via `--no-ai`. Key changes: - Move cmd/ai/security.go → cmd/aws/security.go - Move cmd/ai/compliance.go → cmd/aws/compliance.go - Move pkg/ai/security/ → pkg/aws/security/ - Move schema from ai.security → aws.security (AWSSecuritySettings) - Flip --no-ai flag to --ai (opt-in, default false) - Use error builder pattern for all error handling - Replace errors.Join with fmt.Errorf for order preservation - Add tests for parseOutputFormat, parseSource, parseSeverities, etc. - Add Docusaurus docs under cli/commands/aws/ and cli/configuration/aws/ - Update PRD to v0.3 with aws namespace and --ai flag - Update depguard to allow AWS SDK in pkg/aws/security/ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… status Move --ai from a command-local flag on `atmos aws security` to a global persistent flag available to all commands. Register via GlobalOptionsBuilder with ATMOS_AI env var support. Update PRD to v0.4 with detailed completion status for all phases. Update global-flags.mdx documentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Resolve conflicts: - website/docs/cli/global-flags.mdx: keep main's expanded --ai docs + new --skill flag - pkg/flags/global/flags.go: remove duplicate AI field (keep main's version with Skill) - pkg/flags/global_builder.go: remove duplicate registerAIFlags (keep main's with --skill) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…compliance-1 # Conflicts: # errors/errors.go
…ial validation, tests 1. Wire up --controls flag in compliance command — parses comma-separated control IDs, filters report, recalculates score. 2. Fix compliance total_controls — was set to failing count (score always 0%). Now uses DescribeStandardsControls API with graceful fallback. 3. Add AWS credential validation — early STS GetCallerIdentity check before pipeline starts. Clear error with actionable hints. 4. Add 30+ new tests — all testable functions at 100% coverage. Covers: flag parsing, error sentinels, validation, report building, control filtering, shorthand aliases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add `identity` field to `AWSSecuritySettings` schema and `--identity`/`-i` CLI flag to both `atmos aws security analyze` and `atmos aws compliance report`. When set, credentials are resolved through the Atmos Auth provider chain (SSO → role assumption → isolated credentials), targeting the delegated admin account where Security Hub aggregates all findings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- AWS client cache: newAWSClientCache, WithAuthContext, empty maps - resolveAuthContext: empty identity (nil), non-empty with no auth config (error) - Schema: identity field read/write, empty default - Coverage: WithAuthContext 100%, resolveAuthContext 73%, pkg/aws/security 90.1% Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add `region` field to AWSSecuritySettings for Security Hub aggregation
region. Precedence: --region CLI flag > config region > default (us-east-1).
Combined with `identity`, users configure once in atmos.yaml:
aws:
security:
identity: "security-readonly" # Which account
region: "us-east-2" # Which region
No extra flags needed for subsequent queries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add identity and region fields to the main Configuration section example. Add --identity flag to CLI flags table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two changes ensuring consistent, reproducible AI analysis across all providers: 1. Formalized Remediation schema — added Steps, References fields, changed StackChanges to string. Every output follows the same structure. 2. Created atmos-aws-security agent skill — embedded as system prompt via go:embed. Instructs AI to use exact section headers that map directly to Remediation struct fields. Parser handles both structured and legacy formats via extractFirstMatch fallback. Refactored for complexity compliance: - parseRemediationResponse uses extractFirstMatch helper (17 → 8 complexity) - parseListItems/extractListItem replace parseNumberedList/parseReferenceList - Named constants for magic numbers New tests: skill prompt embedding, parseListItems (6 cases), structured format parsing, fallback format parsing, JSON round-trip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows the complete flow: AWS APIs → Finding → ComponentMapping → Remediation → Report → ReportRenderer (4 formats). Explains behavior with and without --ai flag, and how the skill prompt ensures consistent AI output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Warning This PR exceeds the recommended limit of 10,000 lines.Large PRs are difficult to review and may be rejected due to their size. Please verify that this PR does not address multiple issues. |
|
Important Cloud Posse Engineering Team Review RequiredThis pull request modifies files that require Cloud Posse's review. Please be patient, and a core maintainer will review your changes. To expedite this process, reach out to us on Slack in the |
osterman
reviewed
Apr 5, 2026
…izations API Code organization: - Move cmd/aws/security.go → cmd/aws/security/ subpackage (EKS/ECR pattern) - Move cmd/aws/compliance.go → cmd/aws/compliance/ subpackage - Delete cmd/aws/credentials.go — inline auth directly in each command - Delete cmd/aws/common/ — no shared cmd-level packages needed - Move skill_prompt.md → pkg/aws/security/markdown/ - Move ParseOutputFormat to pkg/aws/security/types.go (next to OutputFormat type) Authentication: - Inline authenticateAndResolveAWS() in each command — calls auth.CreateAndAuthenticateManagerWithAtmosConfig() directly, reads AuthContext.AWS (same pattern as S3 backend) - Move ValidateAWSCredentials to pkg/aws/identity/identity.go (next to GetCallerIdentity) - Use narrow stackInfoProvider interface for extractAWSAuthContext AWS Organizations API: - Add OrganizationsAPI interface and client for account name lookup - resolveAccountName checks config account_map first (no API call), then falls back to DescribeAccount API, caches results - Update example atmos.yaml and config docs — account_map is now optional Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (4)
pkg/aws/security/types_test.go (1)
54-55: Userequire.ErrorIsfor cleaner error assertion.
assert.True(errors.Is(...))works butrequire.ErrorIsis the idiomatic testify pattern for error type assertions and gives tighter diagnostics.♻️ Suggested refactor
import ( - "errors" "testing" @@ _, err := ParseOutputFormat(format) require.Error(t, err) - assert.True(t, errors.Is(err, errUtils.ErrAISecurityInvalidFormat), - "expected ErrAISecurityInvalidFormat, got: %v", err) + require.ErrorIs(t, err, errUtils.ErrAISecurityInvalidFormat) }) } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@pkg/aws/security/types_test.go` around lines 54 - 55, Replace the current assertion that uses assert.True(t, errors.Is(err, errUtils.ErrAISecurityInvalidFormat), ...) with the idiomatic Testify call require.ErrorIs(t, err, errUtils.ErrAISecurityInvalidFormat) to make the test fail fast and provide better diagnostics; locate the line using assert.True + errors.Is and swap it to require.ErrorIs while keeping the same err and errUtils.ErrAISecurityInvalidFormat references.pkg/aws/security/component_mapper.go (2)
586-611:lookupAccountNameusescontext.Background()ignoring caller's context.Lines 591 and 597 create fresh contexts instead of accepting one from the caller. If the parent context is cancelled (e.g., timeout), these AWS API calls will continue running.
Consider threading context through
resolveAccountName→lookupAccountName, or at minimum use the mapper's configured timeout.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@pkg/aws/security/component_mapper.go` around lines 586 - 611, The lookupAccountName function creates fresh contexts with context.Background() for calls to m.clients.getOrganizationsClient and client.DescribeAccount which ignores caller cancellation; change lookupAccountName to accept a context.Context parameter (and update resolveAccountName to pass that ctx through) and replace both context.Background() uses with the passed ctx (or, if a mapper-level timeout context exists on m, use that consistently). Update all callers of resolveAccountName/lookupAccountName to forward their context, and ensure getOrganizationsClient and the DescribeAccount invocation use that ctx so parent cancellations/timeouts are honored.
71-80: Missingperf.Trackon public methodMapFinding.Per coding guidelines, all public functions should have performance tracking. This method is called frequently per-finding and would benefit from instrumentation.
Suggested fix
func (m *dualPathMapper) MapFinding(ctx context.Context, finding *Finding) (*ComponentMapping, error) { + defer perf.Track(nil, "security.dualPathMapper.MapFinding")() + // Path A: Try tag-based mapping first. mapping, err := m.mapByTags(ctx, finding)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@pkg/aws/security/component_mapper.go` around lines 71 - 80, The public method dualPathMapper.MapFinding is missing performance instrumentation; add a perf.Track invocation at the start of MapFinding (e.g., call perf.Track(ctx, "dualPathMapper.MapFinding") and defer the returned stop/cleanup function) so the method's execution time is recorded for every invocation before running mapByTags/mapByHeuristics.cmd/aws/security/security_test.go (1)
443-447: Consider strengthening this assertion.Checking
Args != nilonly confirms the field is set, not what constraint is applied. A command withcobra.ArbitraryArgswould pass but violate the "no args" intent.♻️ Verify actual constraint
func TestSecurityCmdUsesNoArgs(t *testing.T) { // Both security and security analyze commands should accept no positional args. - assert.NotNil(t, SecurityCmd.Args, "SecurityCmd should have Args set") - assert.NotNil(t, securityAnalyzeCmd.Args, "securityAnalyzeCmd should have Args set") + // Verify commands reject positional arguments by checking Args validator behavior. + err := SecurityCmd.Args(SecurityCmd, []string{"unexpected-arg"}) + assert.Error(t, err, "SecurityCmd should reject positional args") + err = securityAnalyzeCmd.Args(securityAnalyzeCmd, []string{"unexpected-arg"}) + assert.Error(t, err, "securityAnalyzeCmd should reject positional args") }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cmd/aws/security/security_test.go` around lines 443 - 447, The test currently only checks Args != nil; update it to assert the actual constraint function equals cobra.NoArgs so the commands truly disallow positional args: replace the NotNil assertions with equality checks comparing SecurityCmd.Args and securityAnalyzeCmd.Args to cobra.NoArgs (i.e., assert.Equal(t, cobra.NoArgs, SecurityCmd.Args, ...) and similarly for securityAnalyzeCmd.Args) so the test fails if a different Args policy (like cobra.ArbitraryArgs) is set.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@cmd/aws/compliance/compliance.go`:
- Around line 163-193: The loop over frameworks currently calls
renderer.RenderComplianceReport() for each report which concatenates multiple
structured outputs (JSON/YAML/CSV) into an invalid stream; instead collect
non-nil reports into a slice (e.g., []security.ComplianceReport or the concrete
report type) while still handling Markdown-to-stdout rendering per-framework,
and after the loop call renderer.RenderComplianceReport once with the aggregate
slice when outputFormat != security.FormatMarkdown or fileOutput != "" so
structured formats are emitted as a single valid document; keep existing
filtering via filterComplianceReport and the fetcher.FetchComplianceStatus/error
handling unchanged, and update any types passed to
renderer.RenderComplianceReport to match the aggregated payload.
- Around line 246-268: filterComplianceReport currently recalculates totals
using the original report.PassingControls and therefore ignores passing controls
that are part of the controlFilter; update the function to recompute passing and
failing counts from the filtered set: iterate report.FailingDetails to count
filtered failing controls (already done into filtered) and also iterate
report.PassingDetails (or if only passing count is available, derive which
control IDs in controlFilter are not in filtered failing set) to count filtered
passing controls, then set filteredReport.FailingControls,
filteredReport.PassingControls, filteredReport.TotalControls = passing+failing
and compute ScorePercent =
float64(filteredReport.PassingControls)/float64(filteredReport.TotalControls)*percentMultiplier
(handle zero total case) so all totals and score reflect only controls in
controlFilter.
---
Nitpick comments:
In `@cmd/aws/security/security_test.go`:
- Around line 443-447: The test currently only checks Args != nil; update it to
assert the actual constraint function equals cobra.NoArgs so the commands truly
disallow positional args: replace the NotNil assertions with equality checks
comparing SecurityCmd.Args and securityAnalyzeCmd.Args to cobra.NoArgs (i.e.,
assert.Equal(t, cobra.NoArgs, SecurityCmd.Args, ...) and similarly for
securityAnalyzeCmd.Args) so the test fails if a different Args policy (like
cobra.ArbitraryArgs) is set.
In `@pkg/aws/security/component_mapper.go`:
- Around line 586-611: The lookupAccountName function creates fresh contexts
with context.Background() for calls to m.clients.getOrganizationsClient and
client.DescribeAccount which ignores caller cancellation; change
lookupAccountName to accept a context.Context parameter (and update
resolveAccountName to pass that ctx through) and replace both
context.Background() uses with the passed ctx (or, if a mapper-level timeout
context exists on m, use that consistently). Update all callers of
resolveAccountName/lookupAccountName to forward their context, and ensure
getOrganizationsClient and the DescribeAccount invocation use that ctx so parent
cancellations/timeouts are honored.
- Around line 71-80: The public method dualPathMapper.MapFinding is missing
performance instrumentation; add a perf.Track invocation at the start of
MapFinding (e.g., call perf.Track(ctx, "dualPathMapper.MapFinding") and defer
the returned stop/cleanup function) so the method's execution time is recorded
for every invocation before running mapByTags/mapByHeuristics.
In `@pkg/aws/security/types_test.go`:
- Around line 54-55: Replace the current assertion that uses assert.True(t,
errors.Is(err, errUtils.ErrAISecurityInvalidFormat), ...) with the idiomatic
Testify call require.ErrorIs(t, err, errUtils.ErrAISecurityInvalidFormat) to
make the test fail fast and provide better diagnostics; locate the line using
assert.True + errors.Is and swap it to require.ErrorIs while keeping the same
err and errUtils.ErrAISecurityInvalidFormat references.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 9111d9d8-dcd0-4a0c-a3c9-dad460f9ae91
📒 Files selected for processing (17)
cmd/aws/aws.gocmd/aws/compliance/compliance.gocmd/aws/compliance/compliance_test.gocmd/aws/compliance/markdown/atmos_aws_compliance.mdcmd/aws/security/markdown/atmos_aws_security.mdcmd/aws/security/security.gocmd/aws/security/security_test.godocs/prd/atmos-aws-security-compliance.mdexamples/aws-security-compliance/atmos.yamlpkg/aws/identity/identity.gopkg/aws/security/analyzer.gopkg/aws/security/aws_clients.gopkg/aws/security/component_mapper.gopkg/aws/security/markdown/skill_prompt.mdpkg/aws/security/types.gopkg/aws/security/types_test.gowebsite/docs/cli/configuration/aws/security.mdx
✅ Files skipped from review due to trivial changes (6)
- cmd/aws/aws.go
- cmd/aws/security/markdown/atmos_aws_security.md
- website/docs/cli/configuration/aws/security.mdx
- examples/aws-security-compliance/atmos.yaml
- cmd/aws/compliance/compliance_test.go
- pkg/aws/security/types.go
🚧 Files skipped from review as they are similar to previous changes (1)
- pkg/aws/security/aws_clients.go
- Use require.ErrorIs instead of assert.True(errors.Is(...)) in types_test - Thread context.Context through resolveAccountName → lookupAccountName and all caller methods (mapByAccountID, mapByECRRepo, mapByResourceType) - Add perf.Track to MapFinding public method - Strengthen Args test to actually invoke Args() validator - Add analyzer tests: direct tool call loop, error path, CLI fallback, nil tool result handling Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
osterman
reviewed
Apr 5, 2026
osterman
reviewed
Apr 5, 2026
- Add wrapAWSServiceError() that detects Security Hub not enabled, access denied, and connection errors — provides actionable hints instead of raw AWS SDK error messages - Add ValidateAWSCredentials tests via mockable getCallerIdentityFn - Reduce example README from 302 to 119 lines — condensed output examples while keeping all 4 scenarios - Add wrapAWSServiceError tests for all error patterns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sage Address Erik's review: - Remove "AWS" from ErrAWSSecurityNotEnabled message text - Rename all ErrAISecurity* → ErrAWSSecurity* error sentinels (45 refs across 14 files) since these are AWS-specific errors Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
osterman
approved these changes
Apr 6, 2026
|
Warning Release Documentation RequiredThis PR is labeled
|
1 similar comment
|
Warning Release Documentation RequiredThis PR is labeled
|
|
These changes were released in v1.215.0-rc.3. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
what
atmos aws security analyze— fetch security findings from AWS Security Hub, map them to Atmos components and stacks via resource tags, and generate structured remediation reportsatmos aws compliance report— generate compliance posture reports against industry frameworks (CIS AWS, PCI-DSS, SOC2, HIPAA, NIST) with pass/fail scoringfinding-tag(exact),tag-api(exact),context-tags(high),account-map(low),ecr-repo(low),naming-convention(low),resource-type(low)--aiflag — multi-turn tool analysis reads component source and stack config, generates root cause analysis, specific code changes, stack YAML changes, deploy commands, risk assessmentidentityconfig field for targeting Security Hub delegated admin accounts--stack,--component,--severity,--source,--framework,--format,--file,--max-findings,--no-group,--region,--identityflagsatmos_list_findings,atmos_describe_finding,atmos_analyze_finding,atmos_compliance_reportexamples/aws-security-compliance/why
Reviewing AWS security findings today requires navigating multiple AWS console pages, cross-referencing resources with Terraform code, and manually figuring out which configuration caused the issue. This is slow, error-prone, and requires deep AWS + Terraform expertise.
Atmos owns the component-to-stack relationship, so it can trace a security finding on an AWS resource all the way back to the exact Terraform code and stack configuration that created it — and generate a targeted fix.
The key differentiator vs AWS MCP security servers: MCP servers return raw findings but have no concept of Atmos stacks, components, or Terraform source code. Our implementation maps findings to IaC and generates specific remediation with deploy commands.
See It in Action
Tested against a multi-account AWS organization (11 accounts, Security Hub delegated admin, 500 findings, 97.2% mapped to Atmos components).
1. Security findings mapped to components
2. AI-powered remediation (
--ai)3. Compliance report
4. Compliance report with AI (
--ai)New CLI Commands
atmos aws security analyzeFetches findings from Security Hub, maps them to Atmos components via resource tags (7 mapping strategies with confidence levels), and renders reports in 4 formats. Post-mapping
--stackand--componentfilters narrow results after mapping. With--ai, the AI reads component source code and stack config via multi-turn tools to generate specific remediation.atmos aws compliance reportQueries Security Hub enabled standards, counts total controls via
ListSecurityControlDefinitions, and computes pass/fail scores. Supports--frameworkfilter for CIS AWS, PCI-DSS, SOC2, HIPAA, NIST. With--ai, generates prioritized remediation for each failing control.Configuration
Example
See
examples/aws-security-compliance/for a complete configuration with auth, tag mapping, AI provider, and all commands.references
docs/prd/atmos-aws-security-compliance.mdwebsite/blog/2026-04-03-aws-security-compliance.mdxexamples/aws-security-compliance/website/docs/cli/commands/aws/security/,website/docs/cli/commands/aws/compliance/website/docs/cli/configuration/aws/security.mdxSummary by CodeRabbit
Release Notes
New Features
--aiflag for structured guidanceDocumentation
atmos aws security analyzeandatmos aws compliance reportcommands